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IMAGE PROCESSING PROGRAM AND IMAGE PROCESSING 

APPARATUS 

BACKGROUND OF THE INVENTION 

Field of the Invention 

5 The present invention relates to an image processing program and an image 

processing apparatus, and more specifically to an image processing program and an 
image processing apparatus capable of extracting a still picture from moving pictures and 
recording the extracted image. 

Description of the Related Art 
10 Conventionally, a still picture is segmented from captured moving pictures, and 

recorded on a record medium as still picture data separately from the original moving 
picture data. 

For example, Japanese Patent Application Publication No. 7-177453 discloses 
an apparatus capable of taking a favorite scene with a shutter key pressed while 
15 confirming moving pictures played back on a screen so that the scene can be printed as a 
still image on the apparatus. 

Japanese Patent Application Publication No. 2000-152165 also proposes 
extracting an output scene at predetermined time intervals from moving pictures recorded 
on a record medium, and outputting the scene as a still picture on another record 
20 medium. 

SUMMARY OF THE INVENTION 
However, in Japanese Patent Application Publication No. 7-177453, only one 
specific scene can be recorded (printed) as a still picture. Therefore, if the images 
before and after the specific image are to be recorded, the corresponding scenes are to be 
25 individually specified. Especially, when a plurality of frames are to be recorded at 

predetermined intervals in continuous shooting, etc., the user who takes the images has to 
specify each image to be recorded by appropriately adjusting the timing, thereby 
requiring a laborious operation. 
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Furthermore, in Japanese Patent Application Publication No. 2000-152165, 
since a still picture is automatically extracted from the head of the moving pictures at 
predetermined intervals, a user cannot extract a desired scene. 

The present invention has been developed to solve the above-mentioned 
5 problems, and aims at providing an image processing program and an image processing 
apparatus capable of easily segmenting images around a favorite scene from moving 
pictures and recording the images. 

To attain the above-mentioned objective, the first aspect of the present invention 
provides an image processing program used to direct a computer to perform the functions 

10 of: playing back moving picture data recorded on a record medium, and displaying 
moving pictures in a moving picture display area set on a monitor; extracting a 
predetermined number of frames of still picture data immediately before, after, or before 
and after a moment when a still picture extract command is input, from moving picture 
data at predetermined intervals, together with still picture data at the moment when the 

15 still picture extract command is input, based on the still picture extract command input 
during playback of the moving picture data; playing back each piece of the extracted still 
picture data, and displaying a still picture in a plurality of still picture display areas set in 
an area different from the moving picture display area on the monitor; and recording 
each piece of the extracted still picture data on a record medium. 

20 According to the first aspect of the present invention, when a still picture extract 

command is input, a predetermined number of frames of still picture data immediately 
before, after, or before and after a moment when the still picture extract command is 
input are extracted from moving picture data at predetermined intervals together with 
still picture data at the moment when the still picture extract command is input, and 

25 recorded on a record medium. The extracted still picture data is played back on the 

monitor together with the moving pictures. At this time, the still picture is displayed on 
a still picture display area different from the moving picture display area in which the 
moving pictures are displayed. 

To attain the above-mentioned objective, the second aspect of the present 

30 invention provides an image processing apparatus which includes: a moving picture 

playback device which plays back moving picture data recorded on a record medium, and 
displays moving pictures in a moving picture display area set on a monitor; a record 
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instruction device which designates a frame of a moving picture displayed in the moving 
picture display area to be recorded as a still picture; a still picture data extraction device 
which extracts a predetermined number of frames of still picture data immediately before, 
after, or before and after a moment when a record instruction is received from moving 
5 picture data at predetermined intervals together with still picture data at the moment 

when the record instruction is received based on the record instruction received from the 
record instruction device; a still picture playback device which plays back each piece of 
the still picture data extracted by the still picture data extraction device, and displays a 
still picture in a plurality of still picture display areas set in an area different from the 
10 moving picture display area on the monitor; and a recording device which records each 
piece of the still picture data extracted by the still picture data extraction device on a 
record medium. 

According to the second aspect of the present invention, when a record 
instruction is received from a record instruction device during playback of moving 

15 pictures, a predetermined number of frames of still picture data immediately before, after, 
or before and after a moment when a record instruction is received are extracted from 
moving picture data at predetermined intervals together with still picture data at the 
moment when the record instruction is received by the still picture data extraction device. 
The extracted still picture data is recorded on a record medium by a recording device 

20 separately from the moving picture data. Thus, images around the selected image can 
be obtained, and can also be segmented as those by continuous shooting. Furthermore, 
the extracted still picture data is played back on the monitor together with the moving 
pictures by a still picture playback device. At this time, the still picture is displayed in 
the still picture display area different from the moving picture display area. Thus, a 

25 glance at the still picture extracted from the moving pictures can allow it to be 
immediately checked. 

To attain the above-mentioned objective, third aspect of the present invention 
provides the image processing apparatus described by referring to the second aspect of 
the present invention. It further includes an extraction condition instruction device 

30 which designates a number of frames and intervals of still picture data to be extracted 
from the moving picture data. The still picture data extraction device extracts the still 
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picture data at the intervals and the number of frames of the still picture data designated 
by the extraction condition instruction device from the moving picture data. 

According to the third aspect of the present invention, the number of frames of 
segmented still pictures and the intervals can be optionally designated. 
5 To attain the above-mentioned objective, the fourth aspect of the present 

invention provides the image processing apparatus which is described by referring to the 
second and third aspects of the present invention. The apparatus further includes a 
record execution instruction device which designates execution of recording each piece 
of still picture data extracted by the still picture data extraction device. The recording 
10 device records on a record medium each piece of still picture data extracted by the still 
picture data extraction device at an instruction to execute recording from the record 
execution instruction device. 

According to the fourth aspect of the present invention, the still picture data 
extracted at an instruction to execute recording from the record execution instruction 
15 device is recorded. Thus, excess recording can be avoided. 

Furthermore, to attain the above-mentioned objective, the fifth aspect of the 
present invention provides the image processing apparatus according to the second, third, 
or fourth aspect. It further includes: an optical image capturing system; an image 
pickup device; a moving picture data generation device which continuously captures 
20 images picked up by the image pickup device through the optical image capturing system 
in a predetermined period, and generates moving picture data; and a moving picture data 
recording device which records moving picture data generated by the moving picture 
data generation device on a record medium. 

According to the fifth aspect of the present invention, the image processing 
25 apparatus itself has the function of recording moving pictures, and can extract and record 
a still picture corresponding to the moving picture data recorded by the image processing 
apparatus. 

As described above, according to the present invention, images around a favorite 
scene of the user can be extracted as continuous shooting during playback of a video file. 
30 An extracted image can also be confirmed on the same screen as the playback screen of 
the moving picture. Thus, an editing process of extracting a still picture as continuous 
shooting from moving pictures and storing the picture can be easily performed. 
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BRIEF DESCRIPTION OF THE DRAWINGS 
Fig. 1 is a front view of an embodiment of a camera to which the present 
invention is applied; 

Fig. 2 is a side view of an embodiment of a camera to which the present 
5 invention is applied; 

Fig. 3 is a rear elevation of an embodiment of a camera to which the present 
invention is applied; 

Fig. 4 is a block diagram of the inner configuration of a camera to which the 
present invention is applied; 
10 Fig. 5 shows the concept of the structure of a still picture file; 

Fig. 6 shows the concept of the structure of a moving picture file; 
Figs. 7 A to 7B show an example of the display of a menu screen during moving 
picture editing; 

Figs. 8 A to 8B show an example of the display of a menu screen during still 
15 picture segmenting edit; 

Fig. 9 shows an example of the display of the display of a still picture 
segmenting edit screen; 

Fig. 10 shows an example of the display of a liquid crystal monitor during still 
picture segmenting edit; 
20 Fig. 1 1 shows an example of the display of a liquid crystal monitor after still 

picture segmenting edit; 

Fig. 12 shows another example of the display of the display of a still picture 
segmenting edit screen; 

Fig. 13 shows another example of the display of the display of a still picture 
25 segmenting edit screen; 

Fig. 14 shows another example of the display of the display of a still picture 
segmenting edit screen; and 

Figs, 15 A to 15C show another example of the display of the display of a still 
picture segmenting edit screen. 



DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 
Preferred embodiments of the image processing program and the image 
processing apparatus are described below in detail by referring to the attached drawings. 

Figs. 1 to 3 are a front view, a side view, and a rear elevation respectively 
showing an embodiment of a camera to which the present invention is applied. 

A camera 10 is a digital camera having the function of recording and playing 
back both still and moving pictures. A camera body 12 has a taking lens 14, a strobe 16, 
a strobe dimmer sensor 18, and a self-timer lamp 20 in front. The camera body 12 has a 
shutter button 22, a zoom lever 24, a strobe button 26, a macro button 28, etc. on the side 
of the camera body 12. It also has a liquid crystal monitor 30, a display button 32, a 
menu/OK button 34, a cross button 36, a BACK button 38, a power button 40, a mode 
switch 42, a speaker 44, etc. at the back. Although not shown in the attached drawings, 
the camera body 12 has a microphone on the top of the camera body 12. 

The mode switch 42 functions as a switch for setting the modes of the camera 10. 
By sliding the mode switch 42, the camera 10 can be set in one of a still mode, a movie 
mode, and a playback mode. In the still mode, a still picture is recorded. In the movie 
mode, a moving picture is recorded. In the playback mode, the recorded still or moving 
picture is played back. 

The shutter button 22 function as a button for designation of capturing a still 
picture in the still mode, and also functions as a button (what is called a recording 
button) for designation of start and stop of capturing a moving picture in the movie mode. 
The shutter button is configured by a two-step switch having a half-press step and a 
full-press step. In the half-press step, the switch SI is turned on. In the full-press step, 
the switch S2 is turned on. 

The zoom lever 24 functions as a button for designation of a zoom 
(telescope/wide view) of a captured image during capturing, and also functions as a 
switch for designation of a zoom (enlarging/reducing) of a displayed image during 
playback. The zoom lever 24 is attached as being freely slid in the vertical direction, 
and a zooming operation can be performed on the telescope (during capturing) or the 
enlargement (during playback) when it is slid upwards, and a zooming operation can be 
performed on the wide view (during capturing) or the reduction (during playback) when 
it is slid downwards. 
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The strobe button 26 functions as a button for selection of the strobe mode under 
the still mode, and the macro button 28 functions as a button for designation of turning 
ON/OFF of the macro function under the still mode. 

The liquid crystal monitor 30 is used as a playback screen of a video file 
5 recorded on a record medium, and also used as an electronic view finder with a real time 
picture (through image) during capturing displayed. It is also used as a display screen 
of a menu when a user performs each setting operation, and as an editing screen of a 
captured image. The display button 32 functions as a button for designation of a 
display switch on the liquid crystal monitor 30. 
1° The menu/OK button 34 functions as a button for designation of a switch from a 

normal screen of each mode to a menu screen, and also as a button for designation of the 
determination (registration) of selected contents and the execution (confirmation) of a 
process. 

The cross button 36 functions as a button for input of the designation of four 
15 directions of upward, downward, right, and left, and is used in selecting a desired item 
from the menu screen, etc. and designating the selection contents of each set item of each 
menu. 

The BACK button 38 functions as a button for designation of a cancellation, etc. 
of an inputting operation, and used when each setting operation, etc. is stopped during 
20 the operation. 

Fig. 4 is a block diagram of the internal configuration of the camera 10 
according to an embodiment of the present invention shown in Figs. 1 to 3. 

In Fig. 4, a CPU 50 is a control unit for integrally controlling the present camera 
system, and integrally controls each circuit according to a predetermined based on the 
25 input from a camera operation unit 54 such as the above-mentioned shutter button 22, 

zoom lever 24, strobe button 26, macro button 28, display button 32, menu/OK button 34, 
cross button 36, BACK button 38, power button 40, mode switch 42, etc. The ROM 52 
stores a program processed by the CPU 50 and various data, etc. required in the control. 
The quantity of light passing through the taking lens 14 is adjusted by a 
30 diaphragm 56, and then input to CCD 58. A photosensor is arranged in a flat array on 
the photoreceptive surface of the CCD 58. A subject image formed on the 



photoreceptive surface of the CCD 58 through the taking lens 14 is converted by each 
photosensor into a signal charge of a quantity depending on the amount of incident light. 

The signal charge stored in each photosensor is sequentially read as a voltage 
signal (video signal) depending on the signal charge based on the pulse provided from a 
5 CCD driver 60, and output to an analog signal processing unit 62. 

The analog signal processing unit 62 includes a signal processing unit such as a 
sampling hold circuit, a color separation circuit, a gain adjustment circuit, etc. In the 
analog signal processing unit 62, a correlative double-sampling (CDS) process and a 
color separating process on each of the R, G, and B color signals are performed, and the 
10 signal level of each color signal is adjusted (pre-white-balance process). The signal 
output from the analog signal processing unit 62 is converted by an A/D converter 64 
into a digital signal, then stored in RAM 68 through a built-in buffer 66, and output to a 
digital signal processing unit 70 from the RAM 68. 

The CCD driver 60, the analog signal processing unit 62, and the A/D converter 
15 64 is provided with a timing signal from a timing generator 67. Using the timing signal, 
each circuit is synchronized. 

The digital signal processing unit 70 is configured by the video signal 
processing unit such as a brightness/color difference signal generation circuit, a gamma 
correction circuit, a sharpness correction circuit, a contrast correction circuit, a white 
20 balance correction circuit, etc., and a digital signal processor (DSP) including a audio 
signal processing unit, and processes a video signal and an audio signal at a command 
from the CPU 50. The video data input to the digital signal processing unit 70 is 
converted into a brightness signal (Y signal) and a color difference signal (Cr, Cb signal), 
and a predetermined process such as gamma correction, etc. is performed, and the result 
25 is stored in the RAM 68. 

The brightness/color difference signal (YC signal) stored in the RAM 68 is 
output to VRAM 76, and output from the VRAM 76 to an encoder 78. The YC signal 
output to the encoder 78 is converted into a signal (for example, a color composite 
picture signal in the NTSC system) in a predetermined system for display together with a 
30 character and a symbol provided from a character generator 80 and graphic data, and 
output to the liquid crystal monitor 30 through a D/A converter 82. Thus, the image 
contents of video data are displayed on the screen of the liquid crystal monitor 30. 
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The video data in the RAM 68 is periodically rewritten by a video signal output 
from the CCD 58, and the video signal generated from the video data is provided for the 
liquid crystal monitor 30, thereby displaying the image input through the CCD 58 on the 
liquid crystal monitor 30 on the real-time basis. A user who takes the images confirms 
5 the image (through image) displayed on the liquid crystal monitor 30 so that the 
capturing angle of view, the focus status, etc. can be confirmed. 

When the still mode is selected by the mode switch 42, a still picture can be 
recorded. If the shutter button 22 is half pressed in this state (SI ON), the AF control is 
performed, and a main subject comes into focus. The AF control is performed by, for 
10 example, contrast AF, and the taking lens 14 is moved such that the components of a 

high frequency of the G signal can be the maximum, and the subject can come into focus. 

Simultaneously, metering is performed, and the subject brightness (captured EV 
value) is obtained. Then, the diaphragm value and the shutter speed (charge 
accumulation time) during capturing is determined based on the obtained captured EV 
15 value. In the metering, for example, the R, G, and B signals are captured, and the 

captured EV value can be obtained based on the total value obtained by totalizing the R, 
G, and B signals. 

If the shutter button 22 is then fully pressed (S20N), then images are captured 
with the determined diaphragm value and a shutter speed with one frame of video data 
20 captured in the RAM 68. The video data captured in the RAM 68 is transmitted to a 
compression/decompression unit 84 through the RAM 68 after the digital signal 
processing unit 70 performs predetermined signal processing. 

The compression/decompression unit 84 compresses video data at an instruction 
from the CPU 50 in a predetermined compression format (for example, JPEG), and the 
25 compressed video data is recorded in a memory card 88 through a card interface 86 as a 
still picture file in a predetermined format. 

Fig. 5 shows the concept of the structure of a still picture file. As shown in Fig. 
5, a still picture file is formed by a header and still picture data. A header stores header 
information such as a total file size, a data format, etc., thumbnail video data, etc. The 
30 still picture data stores the video data compressed in a predetermined compression format 
(for example, JPEG). 
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When the movie mode is selected by the mode switch 42, moving pictures with 
voice can be recorded. If the shutter button 22 is fully pressed (S20N) in this state, 
then a recording operation starts, and the video data is captured in the RAM 68 at a 
predetermined frame rate. The video data captured in the RAM 68 is transmitted to the 
5 compression/decompression unit 84 after predetermined signal processing is performed 
by the digital signal processing unit 70. 

The compression/decompression unit 84 compresses video data at an instruction 
from the CPU 50 in a predetermined compression format (for example, motion JPEG), 
and the compressed video data is recorded in the memory card 88 through a card 
10 interface 86 as a moving picture file (for example, an AVI file) in a predetermined 
format. 

The recording operation terminates after fully pressing the shutter button 22 
again. The audio data in the moving picture is obtained by amplifying a signal from the 
microphone not shown in the attached drawings by an amplifier, and A/D converting the 
15 amplified signal. 

Fig. 6 shows the concept of the structure of a moving picture file. As shown in 
Fig. 5, a moving picture file is formed by a header, moving picture data, and a data 
index. 

A header stores header information such as a total file size, a total number of 
20 frames, a total audio size, etc., the thumbnail video data of the leading frame, etc. 

The moving picture data stores the body of moving picture data. The body of 
moving picture data is recorded with the video data of a still picture forming an image of 
each frame as the video data compressed in a predetermined compression format (for 
example, JPEG) and the corresponding audio data in an interleaved configuration. 
25 In the data index, the information (address, etc.) required in accessing video data 

and audio data in the moving picture data are recorded. The audio data is managed for 
each second according to the index information, and the compressed video data is 
managed for each frame. 

When the mode switch 42 selects the playback mode, the video file recorded in 
30 the memory card 88 can be played back, and the image of the video file last recorded in 
the memory card 88 is displayed on the liquid crystal monitor 30. At this time, when 
the last recorded video file is a moving picture file, the image of the leading frame is 
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displayed. When the right/left key of the cross button 36 is pressed in this state, the 
frames are sequentially advanced and played back. 

The still picture file is played back by first reading the compressed video data 
stored in the still picture file to the RAM 68 at a command from the CPU 50. Then, it is 
5 transmitted from the RAM 68 to the compression/decompression unit 84, and a 

decompressing process is performed. The decompressed video data is transmitted to the 
VRAM 76, and output from the VRAM 76 to the encoder 78. The video data output to 
the encoder 78 is converted together with the character, symbol, and graphic data 
provided by the character generator 80 into a signal in a predetermined display system, 
10 and output to the liquid crystal monitor 30 through the D/A converter 82. Thus, the 
image of the still picture file is played back and displayed on the liquid crystal monitor 
30. 

The moving picture file is played back by reading each frame of the compressed 
video data stored in the moving picture file to the RAM 68 at a command from the CPU 

15 50. Then, it is transmitted sequentially from the RAM 68 to the 

compression/decompression unit 84, and a decompressing process is performed. Each 
frame of the decompressed still picture data is transmitted to the VRAM 76, and output 
from the VRAM 76 to the encoder 78. Each piece of the still picture data output to the 
encoder 78 is converted together with the character, symbol, and graphic data provided 

20 by the character generator 80 into a signal in a predetermined display system, and output 
to the liquid crystal monitor 30 through the D/A converter 82. Thus, still pictures are 
sequentially updated and displayed on the liquid crystal monitor 30, and moving pictures 
are generated. 

The playback of a moving picture file is started when a predetermined operation 
25 button (for example, it is started by pressing the down key of the cross button 36) is 
pressed in the state in which the image of the leading frame is displayed on the liquid 
crystal monitor 30. 

The camera 10 has the moving picture editing function so that a favorite scene 
can be segmented from the captured moving pictures recorded in the memory card 88, 
30 and recorded as a still picture in a separate file. At this time, the camera 10 according 
to the present embodiment can simultaneously record the images around the selected 
scene in addition to the selected scene. 
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Described below is the moving picture editing function of the camera 10, that is, 
the function of segmenting a still picture. 

When an editing operation is performed on a moving picture file, the mode 
switch 42 is first set in the playback mode. 
5 As described above, when the camera 10 is set in the playback mode, the liquid 

crystal monitor 30 displays the image (the image of the leading frame when a moving 
picture file is processed) of the video file last recorded in the memory card 88. In this 
state, the frames are advanced using the left-right key of the cross button 36, and a 
moving picture file on which the editing operation is performed is selected as shown in 
10 Fig. 7A. 

After the selection, the menu/OK button 34 is pressed to call the menu screen on 
the liquid crystal monitor 30. In this example, as shown in Fig. 7B, each menu item is 
displayed as an icon, and an icon corresponding to "moving picture editing" is selected 
from among the icons using the left-right key of the cross button 36. 

15 When the icon "moving picture editing" is selected, the liquid crystal monitor 30 

displays the menu "moving picture editing", and the item "still picture segmenting" is 
selected from the menu "moving picture editing" using the up-down key of the cross 
button 36. In this example, "still picture segmenting edit" and "moving picture 
segmenting edit" are prepared for the menu "moving picture editing". The "moving 

20 picture segmenting edit" is used in segmenting a desired range from moving pictures, and 
storing the result in a file separate from the original moving picture file. 

When the item "still picture segmenting edit" is selected, the liquid crystal 
monitor 30 displays the mode selection screen of "still picture segmenting edit" as shown 
in Fig. 8A. As the mode of still picture segmenting edit, a "single shot mode" for 

25 segmentation of a favorite scene only and a "continuous shooting mode" for 

segmentation of images around a favorite scene as continuous shooting are set. From 
between the two modes, the "continuous shooting" mode is selected using the up-down 
key of the cross button 36. 

When the menu/OK button 34 is pressed after selecting the "continuous shooting 

30 mode", the setting screen of the continuous shooting mode is displayed on the liquid 
crystal monitor 30 as shown in Fig. 8B. On the setting screen, the setting of a 



13 



segmentation condition, that is, "segmentation range", "number of segmented frames", 
and "segmentation interval" are set. 

The "segmentation range" is used in designating the range of segmenting a still 
picture by any of "immediately before", "immediately after", and "before and after". 
5 The "immediately before" is used in recording several frames immediately before a still 
picture extract command is issued. The "immediately after" is used in recording several 
frames immediately after an extract command is issued. The "before and after" is used 
in recording several frames before and after an extract command is issued. 

The "number of segmented frames" is used in designating the number of frames 
10 to be segmented as still pictures. In this example, "3 frames", "5 frames", and "7 
frames" can be designated. 

The "segmentation interval" is used in designating the interval of segmentation 
for still pictures. In this example, "0.1s", "0.2s", and "0.5s" can be designated. 

On the setting screen of the segmentation condition, for example, if the setting 
15 of "segmentation range" is "immediately after", the setting of "number of segmented 

frames" is "3 frames", and the setting of "segmentation interval" is "0.2s", then three still 
pictures are segmented at intervals of 0.2 second after a still picture extract command is 
issued as shown in Fig. 8B. 

The setting of the segmentation condition is performed using the cross button 36. 
20 After the setting, if the icon "OK" displayed on the screen is selected and the menu/OK 
button 34 is pressed, then the set segmentation condition can be determined. 

In this example, it is assumed as the segmentation condition of still pictures that 
the setting of "segmentation range" is "immediately after", the setting of "number of 
segmented frames" is "3 frames", and the setting of "segmentation interval" is "0.2s". 
25 When the settings of the segmentation condition is completed, the liquid crystal 

monitor 30 displays the editing screen for continuous shooting segmentation of still 
pictures as shown in Fig. 9. 

On this still picture segmenting edit screen, the moving pictures to be edited, the 
still pictures segmented from the moving pictures, the playback bar indicating the 
30 playback progress state, and the playback time are displayed as shown in Fig. 9. 

The moving pictures to be edited are played back and displayed in the "moving 
picture display area" provided substantially at the center of the editing screen, and the 
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still pictures segmented from the moving pictures are displayed in the "still picture 

display areas <1>, <2>, and <3>" provided below the "moving picture display area". 

The playback bar is displayed above the "moving picture display area", and the playback 

time is displayed next to the playback bar. 
5 In the initial state, the image of the leading frame in the moving picture file to 

be edited is displayed in the "moving picture display area". If the down key of the cross 

button 36 is pressed in this state, the CPU 50 detects this, and starts editing process 

according to a predetermined editing program. 

First, a moving picture file to be edited is played back and displayed in the 
10 "moving picture display area" of the liquid crystal monitor 30. At this time, the CPU 50 

reads each frame of the compressed video data from the moving picture file to be edited, 

and stores it in a predetermined area A of the RAM 68. 

The compressed video data stored in the area A of the RAM 68 is transmitted to 

the compression/decompression unit 84 at a command from the CPU 50, and a 
15 decompressing process is performed therein. The decompressed still picture data is 

output to the encoder 78 through the VRAM 76, converted into a signal in a 

predetermined display system therein, and output to the liquid crystal monitor 30 through 

the D/A converter 82. 

Thus, each frame of the compressed video data is read from the moving picture 
20 file to be edited, and the compressed video data stored in the area A of the RAM 68 is 

rewritten frame by frame. Thus, the still picture data in the moving picture file is 

updated and displayed frame by frame in the "moving picture display area" of the liquid 

crystal monitor 30, and a moving picture is formed. 

The compressed video data stored in the area A of the RAM 68 is transmitted to 
25 the compression/decompression unit 84, and stored in the area B of the RAM 68 at a 

command from the CPU 50. Plural frames (for example, 10 frames) of compressed 

video data can be accumulated in the area B of the RAM 68. When the area is full, the 

older compressed video data is sequentially deleted and replaced with new compressed 

video data. 

30 Furthermore, the CPU 50 displays the "playback bar" indicating the start of the 

playback of moving pictures and the playback progress state and the "playback time" on 
the liquid crystal monitor 30. That is, when the playback starts, the CPU 50 counts the 
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number of frames which have been played back, calculates the ratio to the total number 
of frames, and displays the progress as the "playback bar" using a bar graph. Thus, the 
ratio of the playback time to the total recording time is displayed by a bar graph, thereby 
immediately indicating the playback progress state. In the example shown in Fig. 9, the 
5 black area of the playback bar is displayed and enlarged depending on the playback 
progress (for example, when a moving picture file for 30 seconds is played back for 15 
seconds, the black area of the playback bar is enlarged up to half the bar). 

Fig. 10 shows an example of the display screen of the liquid crystal monitor 30 
during moving picture editing. The user checks the moving pictures played back and 

10 displayed in the "moving picture display area", and presses the menu/OK button 34 when 
a favorite scene is displayed. 

The menu/OK button 34 functions as a button for input of a still picture extract 
command in the still picture segmenting editing mode. When it is pressed during 
playback of moving pictures, the image at the moment of the press (image displayed in 

15 the "moving picture display area" the moment when the button is pressed) is extracted 
and displayed in the "still picture display area <1>". 0.2 and 0.4 seconds after the 
moment, the images displayed in the "moving picture display area" are sequentially 
displayed in the "still picture display area <2>" and the "still picture display area <3>" 
with the passage of the playback time. 

20 That is, when the menu/OK button 34 is pressed, the compressed video data 

corresponding to the image displayed in the "moving picture display area" is transmitted 
from the area B of the RAM 68 to the compression/decompression unit 84 at the 
command of the CPU 50. Simultaneously, it is stored in the predetermined area C of 
the RAM 68. 

25 The video data processed by the compression/decompression unit 84 in the 

decompressing process is output to the encoder 78, converted into a display signal, and 
output to the liquid crystal monitor 30 through the D/A converter 82. Thus, the image 
(selected image) for which the menu/OK button 34 is pressed is displayed in the "still 
picture display area <1>". 

30 On the other hand, the compressed video data stored in the predetermined area C 

of the RAM 68 is stored to be recorded as a still picture file. 
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In the menu/OK button 34 is pressed, the selection point is displayed on the 
playback bar. 

When the image at the moment when the menu/OK button 34 is pressed is 
displayed in the "still picture display area <1>", the compressed video data 
5 corresponding to the image displayed 0.2 second later in the "moving picture display 
area" is transmitted from the area B of the RAM 68 to the compression/decompression 
unit 84 at the command of the CPU 50. Simultaneously, it is stored in the area C of the 
RAM 68. The video data processed by the compression/decompression unit 84 in the 
decompressing process is output to the liquid crystal monitor 30 through VRAM 76 the 

10 encoder 78 and the D/A converter 82. Thus, the 0.2 second subsequent image to the 
selected image is displayed in the "still picture display area <2>". 

When the 0.2 second subsequent image to the selected image is displayed in the 
"still picture display area <2>", then the compressed video data corresponding to the 
image displayed in the "moving picture display area" 0.2 second later (that is, 0.4 second 

15 after the menu/OK button 34 is pressed) is transmitted from the area B of the RAM 68 to 
the compression/decompression unit 84 at the command of the CPU 50. 
Simultaneously, it is stored in the area C of the RAM 68. The video data processed by 
the compression/decompression unit 84 in the decompressing process is output to the 
liquid crystal monitor 30 through the VRAM 76, the encoder 78, and the D/A converter 

20 82. Thus, the 0.4 second subsequent image to the selected image is displayed in the 
"still picture display area <3>". 

Thus, when the menu/OK button 34 is pressed during playback of moving 
pictures, the image displayed in the "moving picture display area", the 0.2 second 
subsequent image, and the 0.4 second subsequent image are extracted, and sequentially 

25 displayed in the "still picture display areas <1>, <2>, and <3>". 

As described above, when the menu/OK button 34 is pressed during playback of 
moving pictures, the image displayed in the "moving picture display area" the moment 
when the menu/OK button 34 is pressed, and the images displayed in the "moving picture 
display area" 0.2 second and 0.4 second after the moment are displayed in the "still 

30 picture display area <1>", "still picture display area <2>", and "still picture display area 
<3>" respectively. 
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Moving pictures are continuously played back after the image to be recorded as 
a still picture is selected. When the playback of moving pictures is completed, the 
display of the liquid crystal monitor 30 is switched as shown in Fig. 11, and a list of 
extracted images is displayed with indexes. That is, when the playback of moving 
5 pictures is completed, the display of the liquid crystal monitor 30 is cleared, and then the 
three images extracted during playback of moving pictures are displayed with indexes on 
the liquid crystal monitor 30 based on the compressed video data recorded in the area C 
of the RAM 68. The display with indexes is accompanied with the message indicating 
whether or not the still picture is to be recorded. 

10 When the extracted image is recorded as a still picture file separate from the 

moving picture file, the user presses the menu/OK button 34. When it is not recorded, 
the BACK button 38 is pressed. 

When the menu/OK button 34 is pressed, each frame of the video data stored in 
the area C of the RAM 68 at the command of the CPU 50 is sequentially read, and is 

15 recorded as a still picture file in a predetermined format in the memory card 88 through 
the card interface 86. At this time, the header of the still picture file records the 
position data in the moving picture file as additional information, that is, the data 
indicating video data in which frame has been extracted. 

When the BACK button 38 is pressed, the liquid crystal monitor 30 displays an 

20 initial screen (refer to Fig. 9) so that the editing process can be performed again on the 
same moving picture file. Simultaneously, each frame of the video data stored in the 
areas B and C of the RAM 68 is cleared. 

As described above, the camera 10 according to the present embodiment can 
perform an editing operation on a file of captured moving pictures, and plural frames of 

25 favorite scenes in the moving pictures can be extracted at predetermined intervals as 

continuous shooting. At this time, a still picture extract command can be issued while 
playing back moving pictures. When an extract command is issued, an extracted still 
picture is displayed on the same screen as the playback screen of moving pictures. 
Thus, the extracted images can be immediately checked, thereby easily performing an 

30 editing operation. 

According to the present embodiment, when the playback of moving pictures are 
completed for editing, a list of extracted images is displayed with indexes, and an inquiry 
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as to whether or not a still picture file is to be recorded is issued. However, extracted 
frames of images can be automatically recorded as a still picture file without a display 
with indexes after the playback of moving pictures. 

According to the present embodiment, three frames of images are extracted 
5 every 0.2 second from the moment when the menu/OK button 34 is pressed. However, 
when five frames of images are extracted, for example, as shown in Fig. 12, the "still 
picture display areas <1> to <5>" are provided for five frames above and on the right of 
the "moving picture display area". When seven frames of images are extracted, for 
example, as shown in Fig. 13, the "still picture display areas <1> to <7>" are provided 

10 for seven frames below and on both sides of the "moving picture display area". 

When three images are to be extracted, "still picture display areas <1> to 
<3>"can be provided for three frames on the right as shown in Fig. 14. 

As described above, the screen layout in moving picture editing is formed by the 
"moving picture display area" in which moving pictures recorded in the memory card 88 

15 are played back and displayed, and the "still picture display area" in which the still 

pictures extracted from the moving pictures are played back and displayed, wherein the 
"still picture display area" is displayed in an area separate from the "moving picture 
display area", thereby appropriately changing the areas depending on the size, etc. of the 
liquid crystal monitor 30. 

20 It is not always necessary that the number of "still picture display areas" is equal 

to the number of frames of still pictures to be extracted, and at least one is required. 
For example, when five frames of still pictures are to be extracted, the "still picture 
display areas" for only three frames can be accepted. Similarly, when three frames of 
still pictures are to be extracted, the "still picture display areas" for only one frame can 

25 be accepted. 

In this case, the extracted still pictures are scrolled to be played back and 
displayed in the "still picture display area". For example, when five frames of still 
pictures are extracted and displayed in three frames of "still picture display areas" as 
shown in Figs. 15 A to 15C, the first to three frames are sequentially displayed in the 

30 "still picture display area <1>" to "still picture display area <3>" as shown in Fig. 15 A. 
When the fourth frame is displayed, as shown in Fig. 15B, the first image being 
displayed in the "still picture display area <1>" is cleared and the frames are one frame 
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shifted. That is, the image of the second frame is displayed in the "still picture display 
area <1>", the image of the third frame is displayed in the "still picture display area <2>", 
and the newly extracted image of the fourth frame is displayed in the "still picture 
display area <3>". Similarly, when the image of the fifth frame is displayed, the image 
5 of the second frame being displayed in the "still picture display area <1>" is cleared and 
the image of the third frame is displayed as shown in Fig. 15C, the image of the fourth 
frame is displayed in the "still picture display area <2>", and the newly extracted image 
of the fifth frame is displayed in the "still picture display area <3>". 

Thus, when still pictures larger in number than prepared "still picture display 

10 areas" are to be extracted, the extracted still pictures are scrolled and displayed in the 
"still picture display areas". The scrolling direction can be arbitrarily determined. In 
the case of the display style (display style of the screen during moving picture editing) 
shown in Figs. 15 A to 15C, the pictures can be scrolled downwards (in the case of the 
display style shown in Fig. 9, the pictures can be scrolled to right or left). 

15 According to the present embodiment, the case in which the image displayed 

"immediately after" an extract command is issued is to be extracted is explained as an 
example, but the image "immediately before" the command is extracted as follows. 

For example, when three frames of images relating to the image displayed 
"immediately before" an extract command is issued are to be extracted at intervals of 0.2 

20 second, the menu/OK button 34 is pressed during playback of moving pictures, the 
compressed video data corresponding to the image being displayed in the "moving 
picture display area" 0.4 second before the command is issued is transmitted from the 
area B of the RAM 68 to the compression/decompression unit 84 at a command of the 
CPU 50, and stored in a predetermined area C of the RAM 68. 

25 The video data processed by the compression/decompression unit 84 in the 

decompressing process is output to the liquid crystal monitor 30 through the VRAM 76, 
the encoder 78, and the D/A converter 82, and the 0.4 second preceding image is 
displayed in the "still picture display area <1>". On the other hand, the compressed 
video data stored in the area C of the RAM 68 is stored to be recorded as a still picture 

30 file. 

When the 0.4 second preceding image is displayed in the "still picture display 
area <1>", the compressed video data corresponding to the image displayed in the 



20 

"moving picture display area" 0.2 second before the moment when the menu/OK button 
34 is pressed is transmitted from the area B of the RAM 68 to the 
compression/decompression unit 84 at the command of the CPU 50, and stored in the 
area C of the RAM 68. The video data processed by the compression/decompression 
5 unit 84 in the decompressing process is output to the liquid crystal monitor 30 through 
the VRAM 76, the encoder 78 and the D/A converter 82, and the 0.2 second preceding 
image is displayed in the "still picture display area <2>". 

When the 0.2 second subsequent image is displayed in the "still picture display 
area <2>", the compressed video data corresponding to the image displayed in the 

10 "moving picture display area" the moment when the menu/OK button 34 is pressed is 

transmitted from the area B of the RAM 68 to the compression/decompression unit 84 at 
a command of the CPU 50, and simultaneously stored in the area C of the RAM 68. 
The video data processed by the compression/decompression unit 84 in the 
decompressing process is output to the liquid crystal monitor 30 through the VRAM 76, 

15 the encoder 78, and the D/A converter 82. Thus, the image at the moment when the 
menu/OK button 34 is pressed is displayed in the "still picture display area <3>". 

Thus, when three frames of images relating to the image displayed "immediately 
before" an extract command is issued are to be extracted at intervals of 0.2 second, and 
when the menu/OK button 34 is pressed, the 0.4 second preceding image, the 0.2 second 

20 preceding image, and the image displayed the moment when the button is pressed are 

extracted, and sequentially displayed in the "still picture display area <1> to <3>". The 
similar processes are performed in the case in which five or seven frames of still pictures 
are extracted. 

When the playback of moving pictures is completed, a list of extracted images is 
25 displayed with indexes, an inquiry as to whether or not the still pictures are to be 
recorded is issued, and the pictures are recorded as described by referring to the 
embodiments above. 

When the image displayed "before and after" the moment when the extract 
command is issued is extracted, the following process is performed. 
30 For example, when three frames of images relating to the image displayed 

"before and after" an extract command is issued are to be extracted at intervals of 0.2 
second, the menu/OK button 34 is pressed during playback of moving pictures, then the 
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compressed video data corresponding to the image being displayed in the "moving 
picture display area" 0.2 second before the command is issued is transmitted from the 
area B of the RAM 68 to the compression/decompression unit 84 at a command of the 
CPU 50, and stored in a predetermined area C of the RAM 68. 
5 The video data processed by the compression/decompression unit 84 in the 

decompressing process is output to the liquid crystal monitor 30 through the VRAM 76, 
the encoder 78, and the D/A converter 82, and the 0.2 second preceding image is 
displayed in the "still picture display area <1>". On the other hand, the compressed 
video data stored in the area C of the RAM 68 is stored to be recorded as a still picture 
10 file. 

When the 0.2 second preceding image is displayed in the "still picture display 
area <1>", the compressed video data corresponding to the image displayed in the 
"moving picture display area" at the moment when the menu/OK button 34 is pressed is 
transmitted from the area B of the RAM 68 to the compression/decompression unit 84 at 
15 the command of the CPU 50, and stored in the area C of the RAM 68. The video data 
processed by the compression/decompression unit 84 in the decompressing process is 
output to the liquid crystal monitor 30 through the VRAM 76, the encoder 78 and the 
D/A converter 82, and the 0.2 second preceding image is displayed in the "still picture 
display area <2>". 

20 When the image displayed at the moment when the menu/OK button 34 is 

pressed is displayed in the "still picture display area <2>", then the compressed video 
data corresponding to the image displayed in the "moving picture display area" 0.2 
second after the moment when the menu/OK button 34 is pressed is transmitted from the 
area B of the RAM 68 to the compression/decompression unit 84 at a command of the 

25 CPU 50, and simultaneously stored in the area C of the RAM 68. The video data 
processed by the compression/decompression unit 84 in the decompressing process is 
output to the liquid crystal monitor 30 through the VRAM 76, the encoder 78, and the 
D/A converter 82. Thus, the image 0.2 second after the moment when the menu/OK 
button 34 is pressed is displayed in the "still picture display area <3>". 

30 Thus, when three frames of images relating to the image displayed "before and 

after" an extract command is issued are to be extracted at intervals of 0.2 second, and 
when the menu/OK button 34 is pressed, the 0.2 second preceding image, the image at 



22 

the moment when the button is pressed, and the 0.2 second subsequent image are 
extracted, and sequentially displayed in the "still picture display area <1> to <3>". The 
similar processes are performed in the case in which five or seven frames of still pictures 
are extracted. 

5 When the playback of moving pictures is completed, a list of extracted images is 

displayed with indexes, an inquiry as to whether or not the still pictures are to be 
recorded is issued, and the pictures are recorded as described by referring to the 
embodiments above. 

With the camera according to the present embodiment, a playback bar indicating 

10 the playback progress state is displayed on the screen when the moving picture editing 
operation is performed, and the selection point at which a still picture is segmented is 
displayed on the playback bar. However, the display is arbitrarily performed. When it 
is displayed, the playback state of moving pictures can be easily checked, and 
simultaneously the extraction point of a still picture can be quickly checked. On the 

15 other hand, when it is not displayed, the limited space of the display screen on the liquid 
crystal monitor 30 can be efficiently used by increasing the number of still picture 
display areas and enlarging the areas. 

The moving picture file from which a still picture is extracted can store the 
information about the extracted still picture in the header portion. For example, the 

20 thumbnail video data of the still picture obtained at the moment when the menu/OK 
button 34 is pressed is generated and recorded in the header, and the thumbnail video 
data can be processed as a representative image of the moving pictures. Otherwise, the 
thumbnail video data obtained at the moment when the menu/OK button 34 is pressed 
and the position information (about the image in which frame), etc. can be recorded in 

25 the header so that the playback can be started at the moment when the menu/OK button 
34 is pressed. 

According to the present embodiment, a still picture extract command is input 
when the menu/OK button 34 is pressed. However, it can be input from other buttons, 
for example, a shutter button. 
30 The method for extracting a still picture from moving pictures, and 

recording/playing back the picture is not limited to the above-mentioned embodiments, 
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but can be any method of extracting a still picture from moving pictures for 
recording/playing back the picture using other well known technology. 

In a series of above-mentioned embodiments, the present invention is applied to 
a digital still image camera, but the application of the present invention is not limited to 
this. For example, the present invention can be applied to a digital video camera 
provided with a liquid crystal monitor, a PDA, etc. Furthermore, the moving picture 
editing process and the playing back process according to the present invention can be 
provided as application software for a personal computer so that the moving picture 
editing process and the playing back process can be performed on the personal computer. 



